Skip to content

[CMAKE] Upgrade to stlport 4.6.2 #1237

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

aliendroid1
Copy link

@aliendroid1 aliendroid1 commented Jul 7, 2025

Will upgrade to 5.2.1 soon but upgrading to 4.6.2 first makes it easier to make the changes needed for stlport 5.2.1 to work such as upgrading from the old c iostreams to the standard iostreams.

No need to apply the diff because the changes the diff made (mostly add pragma once) are already a part of stlport 4.6.2

STLport 4.6.2 / 4.6 Changelog

Changes in 4.6.2 since 4.6.1
  • threads.h:447: Fixed _NOTHREADS compilation error (thanks Wu Yongwei)
  • stl_solaris.h: Fixed pthreads compilation error
  • stl_msvc.h: Added #define _STLP_HAS_SPECIFIC_PROLOG_EPILOG for VC7.1
  • locale_impl.cpp: Fixed locale::classic() initialization
Changes in 4.6 since 4.5.3
  • A few ANSI conformance fixes (thanks Richard Peng)
  • Performance optimization for string::reserve (thanks Marc)
  • ICC patches applied (thanks Blaine Kubesh)
  • _thread.c, _thread.h: Merged thread redesign from CVS (thanks Petr Ovchenkov)
  • iostream.cpp, locale_impl.cpp: Static initialization made robust for dynamic loader circular dependencies (thanks Alexey Sarytchev)
  • _function.h: New workaround for mem_fun_* family for compilers with return void bug (thanks Pavel Kuznetsov)
  • stl_solaris.h: v8plus compilation fix (thanks Mukesh Kapoor)
  • num_put_float.cpp: FreeBSD fix
  • _construct.h: Fixed case for compilers with default integer constructor bug
  • fstream.cpp: Text file stream position bug correction (thanks Christopher Kohlert)
  • _num_put.c: Correction of display of min 64-bit signed integer values, e.g. numeric_limits<__int64>::min() on WIN32 (thanks Anders Dalvander)
  • _num_put.c: Add '+' sign for unsigned integer display (thanks Francois Dumont)
  • debug: Many missing iterator invalidation corrected (thanks Detlev V.Davidson and Francois Dumont)
  • _new.h: New config compiler switch for those that define bad_alloc but do not throw it! (thanks ?)
  • _new.h: Changed class nothrow_t to struct nothrow_t (thanks Francois Dumont)
  • _bvector.h: Added |= and &= operators to _Bit_reference struct (thanks Ed Rice)
  • Performance bug in type_traits.h fixed (thanks Francois Dumont)
  • Added Linux spin lock code
  • Added GCC-3.x configurations
  • Fixed HP aCC support
  • Added OpenWatcom support
  • Version numbers bumped
  • Iostreams ported to EMX (gcc on OS/2) (thanks Martin Schaffoener)
  • Iostreams ported to CRAY Unicos C90, T90, and J90 (thanks Geir Johansen)
  • stl_sunpro.h: Fixed mbstate definition for SUN 4.2 (thanks lengzq)
  • complex.cpp, _complex.h: template<> used for specializations
  • _num_put.c: Removed unused locale variable (thanks Petr Ovchenkov)
  • _algo.h: __reverse beautified (thanks Kabanov)
  • dll_main.cpp: force_link() forced to be linked in
  • stl_msvc.h: Config changes for .NET
  • _auto_ptr.h: "struct" changed to "class"
  • _istream.c: M_read_unbuffered fixed (thanks Stefan Schwarzer)
  • stl_gcc.h: Added missing inclusion for SCO platform (thanks Emmanuel Soden)
  • _tree.c: Optimized insert_unique (thanks Timothy)
  • _algo.c: Relaxed type requirements for lower_bound, upper_bound, binary_search
  • _algo.h: adjacent_find() expressed with compare-function flavor
  • cpp_runtime/typeinfo: Adjustments for .NET (thanks Daniel)
  • cstd/cassert: Guard removed (thanks Evan Cheng)

TODO

  • Test against many replays in VC6
  • Test against many save games
  • Check diff in unassemblize

@xezon
Copy link

xezon commented Jul 7, 2025

What is the benefit of this? Will it break Retail compatibility? The plan is to abandon STL Port.

@aliendroid1
Copy link
Author

What is the benefit of this? Will it break Retail compatibility? The plan is to abandon STL Port.

It is only for vc6 and it doesn't break compatibility. It makes vc6 have more standard library features such as the iostreams and improves thread safety and much more. It will reduce the need for a lot of the macros and compat headers needed to make the code work with both vc6 and vc17.

@sorcerer86pt
Copy link

No mismatch with generals or zero hour
For zero hour tested with golden replay 1 plus others.

All details on the https://discord.com/channels/951133504605917224/1391886820043784282 slack thread

@xezon
Copy link

xezon commented Jul 8, 2025

@tomsons26 Do you have any objections about this change?

@xezon xezon requested a review from tomsons26 July 8, 2025 07:00
@xezon
Copy link

xezon commented Jul 8, 2025

tomsons26:

I don't know anything about stls, you want @OmniBlade
That said this ignores the fact theres modifications done by EA to stlport
While minor they def would have some impacts
The stlport versions need diffing to understand implications of the changes made in later version
Furthermore no guarantees the sourceforge link works a month from now

The problem with testing a stl change is regular replays tests aren't enough anymore
The stl changes need to be considered individually and their impact on specific code
AFAIK EA didn't update stlport until RA3 if ever
For likely this exact reason
Its not a slap dash change and done as you'd want to believe

@xezon xezon requested review from OmniBlade and removed request for tomsons26 July 8, 2025 07:36
@xezon
Copy link

xezon commented Jul 8, 2025

It makes vc6 have more standard library features such as the iostreams and improves thread safety and much more. It will reduce the need for a lot of the macros and compat headers needed to make the code work with both vc6 and vc17.

The VC6 compat headers will also be removed when VC6 support is removed.

Was there any observed game/tool issue with STLPort 4.5.3 that this change fixes? Or do the STLPort fixes only have a theoretical benefit?

Given the criticality of STLPort for compatibility, we need to test this change against a few hundred replays.

@xezon xezon added Major Severity: Minor < Major < Critical < Blocker Build Anything related to building, compiling labels Jul 8, 2025
@aliendroid1
Copy link
Author

tomsons26:

I don't know anything about stls, you want @OmniBlade
That said this ignores the fact theres modifications done by EA to stlport
While minor they def would have some impacts
The stlport versions need diffing to understand implications of the changes made in later version
Furthermore no guarantees the sourceforge link works a month from now

As i mentioned already in the main pr comment, the diff was mainly adding pragmas and the newer version already uses pragma once everywhere. The other changes in the diff were adding commented out code. Other than that there were one or two things that the game code itself doesn't use anymore. The rest of the changes are in the changelogs

@xezon
Copy link

xezon commented Jul 8, 2025

This change could perhaps also sneakily break save games. So that would need to be tested as well.

I am concerned about this change. It looks very risky.

@aliendroid1
Copy link
Author

This change could perhaps also sneakily break save games. So that would need to be tested as well.

I am concerned about this change. It looks very risky.

Then test it well (:

@aliendroid1
Copy link
Author

It makes vc6 have more standard library features such as the iostreams and improves thread safety and much more. It will reduce the need for a lot of the macros and compat headers needed to make the code work with both vc6 and vc17.

The VC6 compat headers will also be removed when VC6 support is removed.

Was there any observed game/tool issue with STLPort 4.5.3 that this change fixes? Or do the STLPort fixes only have a theoretical benefit?

Given the criticality of STLPort for compatibility, we need to test this change against a few hundred replays.

whats the relevance if things that will happen when neither version of stlport well be being used

@helmutbuhler
Copy link

I tested this with some replays as well. No mismatches.

@xezon
Copy link

xezon commented Jul 8, 2025

I glanced over it with unassemblize and what it tells is that it introduces massive assembler changes in many functions. From that angle it is risky.

4255 of 19347 functions have less than 100% match

1929 of 19347 functions are not even matching by their mangled name any longer

image

@aliendroid1
Copy link
Author

I glanced over it with unassemblize and what it tells is that it introduces massive assembler changes in many functions. From that angle it is risky.

If it doesn't cause mismatches in 99.9% of games then what difference does it make. Plus it only matters if it changes the deterministic output of functions used for game simulation.

Stlport was the goto stl implementation at the time. You think they would make a minor release that just breaks so much code without even documenting it. A library update, especially a core c++ library, isn't going to make a change all of a sudden like make push_back() add elements in the middle of the vector. You should only check if floating point functions used by our code potentially have changed in such a significant way that they could be optimized differently

@xezon
Copy link

xezon commented Jul 8, 2025

Yes the floating point functions are the worry. Since so much code is mutated everywhere, it could be the case that 1 math somewhere produces a different result. Do we want to take this risk for 0 benefits?

@aliendroid1
Copy link
Author

Yes the floating point functions are the worry. Since so much code is mutated everywhere, it could be the case that 1 math somewhere produces a different result. Do we want to take this risk for 0 benefits?

It's not 0 benefits. It gives us many more modern features in vc6 and makes the code compile with vc7 and vc8. It allows us to significantly reduce code fragmentatuon.

It will reduce our usage of macros and global compat headers for splitting implementations between vc6 code and vc2022.

-For example it will give us containers like unordered_map and unordered_set meaning we don't have to use the non standard hash_map anymore.

  • It will give us the new standard iostreams so we will no longer have to pollute the global namespace with things other using std::cout
  • It will improve performance due to performance improvement made to algorithms
  • It adds more and better debug functionality for things like detecting memory leaks and memory access violations.
  • It fixes bugs in the version of stlport we use right now
  • It will make it easier to investigate which parts lead to incompatibility because the code will compile with vc7
  • It might make it possible to upgrade to vc7, vc7.1 and even vc8 without breaking compatibility if we can adjust them to use the and optimization settings

Making the code less fragmented, more standard, less buggy, easier to debug, more performant, and especially more readable and therefore maintainable are pretty important benefits

Potentially being able to upgrade to a newer version of visual studio without breaking compatibility is a pretty big pro.

Perhaps we'll be to identify the core compatibility issues and how to mitigate them and we'll be able to upgrade all the way to vs2022 eventually without breaking compatibility

@xezon
Copy link

xezon commented Jul 8, 2025

I will leave it to OmniBlade to decide whether we want to go forward with touching STL Port. I do agree with tomsons26 that we should not source this from sourceforge but have our own copy for it.

@aliendroid1
Copy link
Author

I will leave it to OmniBlade to decide whether we want to go forward with touching STL Port. I do agree with tomsons26 that we should not source this from sourceforge but have our own copy for it.

Ominblade already said the following in dev general

"The main benefits I see of @aliendroid1 STLPort upgrades is that it allows us to remove some of the ifdef stuff we have now around containers without waiting until we drop retail. If our current replay testing suite passes on the upgrade then I think we are safe that it hasn't changed any logic. If @aliendroid1 wants to do this work I don't see any reason to say not to with the caveat that if it breaks compat it won't be mergable? Seem reasonable"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Build Anything related to building, compiling Major Severity: Minor < Major < Critical < Blocker
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants